Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC1

We survey some recent research directions within the field of approximate dynamic programming (ADP), with a particular emphasis on rollout algorithms and model predictive control (MPC). We argue that while motivated by different concerns, these two methodologies are closely connected, and the mathematical essence of their desirable properties (cost improvement and stability, respectively) is couched on the central dynamic programming idea of policy iteration. In particular, among other things, we show that the most common MPC schemes can be viewed as rollout algorithms and are related to policy iteration methods. Furthermore, we embed rollout and MPC within a new unifying suboptimal control framework, based on a concept of restricted or constrained structure policies, which contains these schemes as special cases. 1 Research supported by NSF Grant ECS-0218328. Many thanks are due to Janey Yu for helpful comments. 2 Dept. of Electrical Engineering and Computer Science, M.I.T., Cambridge, Mass., 02139. 1